chore(ci): add final_status property on junit XML [APMSP-2610]#1681
chore(ci): add final_status property on junit XML [APMSP-2610]#1681gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits intomainfrom
Conversation
BenchmarksComparisonBenchmark execution time: 2026-03-06 16:39:34 Comparing candidate commit 7b48a91 in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 56 metrics, 2 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | 160.467µs | 161.071µs ± 0.212µs | 161.038µs ± 0.104µs | 161.163µs | 161.477µs | 161.596µs | 162.404µs | 0.85% | 1.639 | 7.471 | 0.13% | 0.015µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | [161.042µs; 161.101µs] or [-0.018%; +0.018%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | 240.351ns | 251.439ns ± 12.908ns | 246.444ns ± 3.417ns | 251.228ns | 286.060ns | 290.613ns | 291.236ns | 18.18% | 1.883 | 2.465 | 5.12% | 0.913ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | [249.650ns; 253.228ns] or [-0.711%; +0.711%] | None | None | None |
Group 3
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | 2.303µs | 2.375µs ± 0.022µs | 2.373µs ± 0.005µs | 2.382µs | 2.420µs | 2.428µs | 2.433µs | 2.55% | -0.598 | 2.815 | 0.93% | 0.002µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | [2.372µs; 2.378µs] or [-0.129%; +0.129%] | None | None | None |
Group 4
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | 534.288µs | 535.650µs ± 1.082µs | 535.286µs ± 0.369µs | 535.760µs | 537.641µs | 539.534µs | 540.744µs | 1.02% | 1.933 | 4.418 | 0.20% | 0.077µs | 1 | 200 |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | 1849302.715op/s | 1866897.459op/s ± 3757.063op/s | 1868160.917op/s ± 1289.283op/s | 1869301.774op/s | 1870378.470op/s | 1870910.421op/s | 1871648.098op/s | 0.19% | -1.917 | 4.316 | 0.20% | 265.664op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | 380.775µs | 381.567µs ± 0.293µs | 381.548µs ± 0.206µs | 381.755µs | 382.032µs | 382.259µs | 382.341µs | 0.21% | 0.155 | -0.175 | 0.08% | 0.021µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | 2615468.083op/s | 2620769.930op/s ± 2013.108op/s | 2620899.233op/s ± 1415.318op/s | 2622276.093op/s | 2623996.214op/s | 2625376.513op/s | 2626219.836op/s | 0.20% | -0.151 | -0.175 | 0.08% | 142.348op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | 189.626µs | 190.179µs ± 0.193µs | 190.156µs ± 0.125µs | 190.285µs | 190.505µs | 190.674µs | 190.883µs | 0.38% | 0.478 | 0.717 | 0.10% | 0.014µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | 5238819.654op/s | 5258212.763op/s ± 5331.962op/s | 5258850.393op/s ± 3449.162op/s | 5262058.792op/s | 5265495.284op/s | 5268502.003op/s | 5273526.316op/s | 0.28% | -0.470 | 0.707 | 0.10% | 377.027op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | 37.391µs | 37.534µs ± 0.060µs | 37.535µs ± 0.037µs | 37.568µs | 37.639µs | 37.662µs | 37.733µs | 0.53% | 0.233 | 0.292 | 0.16% | 0.004µs | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | throughput | 26501850.397op/s | 26642272.440op/s ± 42240.223op/s | 26641805.213op/s ± 25981.077op/s | 26669703.034op/s | 26710690.124op/s | 26729596.060op/s | 26744233.434op/s | 0.38% | -0.222 | 0.280 | 0.16% | 2986.835op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | 45.776µs | 45.918µs ± 0.058µs | 45.913µs ± 0.034µs | 45.950µs | 46.019µs | 46.076µs | 46.170µs | 0.56% | 0.764 | 1.517 | 0.13% | 0.004µs | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | 21659205.872op/s | 21777801.862op/s ± 27470.706op/s | 21780531.862op/s ± 16323.384op/s | 21796392.031op/s | 21815224.734op/s | 21826155.620op/s | 21845577.496op/s | 0.30% | -0.753 | 1.484 | 0.13% | 1942.472op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | [535.500µs; 535.800µs] or [-0.028%; +0.028%] | None | None | None |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | [1866376.766op/s; 1867418.151op/s] or [-0.028%; +0.028%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | [381.527µs; 381.608µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | [2620490.933op/s; 2621048.928op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | [190.152µs; 190.206µs] or [-0.014%; +0.014%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | [5257473.805op/s; 5258951.722op/s] or [-0.014%; +0.014%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | [37.526µs; 37.543µs] or [-0.022%; +0.022%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | throughput | [26636418.351op/s; 26648126.529op/s] or [-0.022%; +0.022%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | [45.910µs; 45.926µs] or [-0.017%; +0.017%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | [21773994.686op/s; 21781609.037op/s] or [-0.017%; +0.017%] | None | None | None |
Group 5
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | 4.947µs | 5.035µs ± 0.057µs | 5.037µs ± 0.042µs | 5.067µs | 5.149µs | 5.157µs | 5.162µs | 2.49% | 0.566 | -0.488 | 1.14% | 0.004µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | [5.027µs; 5.043µs] or [-0.158%; +0.158%] | None | None | None |
Group 6
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | 145.344µs | 147.443µs ± 2.004µs | 147.101µs ± 0.392µs | 147.547µs | 149.170µs | 154.846µs | 168.411µs | 14.49% | 6.924 | 62.760 | 1.36% | 0.142µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | [147.165µs; 147.721µs] or [-0.188%; +0.188%] | None | None | None |
Group 7
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | 196.954ns | 199.325ns ± 1.811ns | 199.310ns ± 1.299ns | 200.414ns | 202.114ns | 206.087ns | 207.662ns | 4.19% | 1.154 | 2.716 | 0.91% | 0.128ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | [199.074ns; 199.576ns] or [-0.126%; +0.126%] | None | None | None |
Group 8
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | 10.607ms | 10.639ms ± 0.015ms | 10.639ms ± 0.008ms | 10.647ms | 10.660ms | 10.679ms | 10.718ms | 0.75% | 1.230 | 5.151 | 0.14% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | [10.637ms; 10.641ms] or [-0.019%; +0.019%] | None | None | None |
Group 9
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | 3.893µs | 3.912µs ± 0.003µs | 3.912µs ± 0.001µs | 3.914µs | 3.917µs | 3.920µs | 3.920µs | 0.20% | -1.585 | 15.471 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/ | throughput | 255096414.169op/s | 255591260.110op/s ± 167953.305op/s | 255604728.688op/s ± 76657.007op/s | 255676965.159op/s | 255820545.460op/s | 255882244.038op/s | 256859304.110op/s | 0.49% | 1.615 | 15.700 | 0.07% | 11876.092op/s | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | 78.966µs | 79.547µs ± 0.348µs | 79.550µs ± 0.273µs | 79.740µs | 80.182µs | 80.386µs | 80.759µs | 1.52% | 0.591 | -0.013 | 0.44% | 0.025µs | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | 12382458.773op/s | 12571393.546op/s ± 54883.432op/s | 12570722.162op/s ± 43065.597op/s | 12619160.011op/s | 12643600.074op/s | 12661707.554op/s | 12663726.404op/s | 0.74% | -0.570 | -0.059 | 0.44% | 3880.845op/s | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | execution_time | 72.343µs | 73.045µs ± 0.395µs | 72.982µs ± 0.286µs | 73.343µs | 73.770µs | 74.027µs | 74.083µs | 1.51% | 0.503 | -0.468 | 0.54% | 0.028µs | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | throughput | 13498402.705op/s | 13690674.944op/s ± 73923.707op/s | 13702012.842op/s ± 53845.807op/s | 13751464.583op/s | 13791162.826op/s | 13821518.013op/s | 13823129.526op/s | 0.88% | -0.483 | -0.495 | 0.54% | 5227.195op/s | 1 | 200 |
| credit_card/is_card_number/37828224631 | execution_time | 3.896µs | 3.913µs ± 0.003µs | 3.913µs ± 0.002µs | 3.915µs | 3.917µs | 3.919µs | 3.921µs | 0.21% | -1.044 | 8.474 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/37828224631 | throughput | 255045177.934op/s | 255560296.307op/s ± 171792.129op/s | 255577823.449op/s ± 120662.506op/s | 255674453.084op/s | 255770120.105op/s | 255816307.663op/s | 256702136.103op/s | 0.44% | 1.063 | 8.606 | 0.07% | 12147.538op/s | 1 | 200 |
| credit_card/is_card_number/378282246310005 | execution_time | 69.156µs | 69.653µs ± 0.336µs | 69.578µs ± 0.226µs | 69.868µs | 70.264µs | 70.596µs | 71.000µs | 2.04% | 1.018 | 1.213 | 0.48% | 0.024µs | 1 | 200 |
| credit_card/is_card_number/378282246310005 | throughput | 14084523.573op/s | 14357153.804op/s ± 68931.528op/s | 14372404.285op/s ± 46754.959op/s | 14413201.071op/s | 14436355.870op/s | 14454060.731op/s | 14460126.653op/s | 0.61% | -0.987 | 1.092 | 0.48% | 4874.195op/s | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | execution_time | 52.154µs | 52.222µs ± 0.035µs | 52.219µs ± 0.021µs | 52.239µs | 52.293µs | 52.306µs | 52.321µs | 0.20% | 0.650 | -0.046 | 0.07% | 0.003µs | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | throughput | 19112908.211op/s | 19148887.822op/s ± 12960.799op/s | 19150230.466op/s ± 7842.632op/s | 19158374.122op/s | 19167009.647op/s | 19169562.879op/s | 19173804.748op/s | 0.12% | -0.647 | -0.051 | 0.07% | 916.467op/s | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | execution_time | 6.028µs | 6.037µs ± 0.008µs | 6.036µs ± 0.002µs | 6.039µs | 6.043µs | 6.074µs | 6.115µs | 1.30% | 5.622 | 42.086 | 0.14% | 0.001µs | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | throughput | 163542966.399op/s | 165641966.296op/s ± 226398.629op/s | 165676175.768op/s ± 66304.030op/s | 165740850.977op/s | 165816877.796op/s | 165870965.361op/s | 165886572.628op/s | 0.13% | -5.571 | 41.400 | 0.14% | 16008.801op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | execution_time | 3.895µs | 3.913µs ± 0.003µs | 3.913µs ± 0.001µs | 3.914µs | 3.916µs | 3.919µs | 3.921µs | 0.20% | -1.544 | 12.363 | 0.06% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | throughput | 255062939.475op/s | 255573395.557op/s ± 164615.501op/s | 255580448.096op/s ± 91177.118op/s | 255663276.661op/s | 255789454.297op/s | 255880314.591op/s | 256761615.223op/s | 0.46% | 1.568 | 12.544 | 0.06% | 11640.074op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | 64.181µs | 64.491µs ± 0.141µs | 64.478µs ± 0.092µs | 64.584µs | 64.756µs | 64.842µs | 64.989µs | 0.79% | 0.541 | 0.324 | 0.22% | 0.010µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | 15387176.879op/s | 15506173.403op/s ± 33797.664op/s | 15509084.202op/s ± 22163.362op/s | 15528898.615op/s | 15556388.604op/s | 15567987.289op/s | 15580942.949op/s | 0.46% | -0.527 | 0.299 | 0.22% | 2389.856op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | 57.836µs | 58.097µs ± 0.159µs | 58.066µs ± 0.100µs | 58.184µs | 58.414µs | 58.515µs | 58.598µs | 0.92% | 0.889 | 0.263 | 0.27% | 0.011µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | 17065408.653op/s | 17212844.645op/s ± 47125.244op/s | 17221776.967op/s ± 29711.801op/s | 17249356.321op/s | 17270138.826op/s | 17283089.560op/s | 17290132.521op/s | 0.40% | -0.877 | 0.235 | 0.27% | 3332.258op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | 3.893µs | 3.912µs ± 0.003µs | 3.912µs ± 0.001µs | 3.914µs | 3.917µs | 3.918µs | 3.919µs | 0.17% | -1.593 | 13.962 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | 255171132.311op/s | 255597758.024op/s ± 173141.847op/s | 255606959.072op/s ± 93756.892op/s | 255696931.649op/s | 255830844.686op/s | 255871390.119op/s | 256882429.824op/s | 0.50% | 1.620 | 14.182 | 0.07% | 12242.977op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | 54.581µs | 54.997µs ± 0.290µs | 54.942µs ± 0.212µs | 55.192µs | 55.526µs | 55.770µs | 55.808µs | 1.58% | 0.708 | -0.207 | 0.53% | 0.020µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | 17918458.020op/s | 18183425.246op/s ± 95396.291op/s | 18200900.959op/s ± 70404.646op/s | 18260873.149op/s | 18305695.348op/s | 18316252.498op/s | 18321500.865op/s | 0.66% | -0.688 | -0.246 | 0.52% | 6745.536op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | 52.151µs | 52.226µs ± 0.035µs | 52.220µs ± 0.021µs | 52.245µs | 52.298µs | 52.318µs | 52.321µs | 0.19% | 0.609 | 0.046 | 0.07% | 0.002µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | 19112727.464op/s | 19147673.920op/s ± 12708.654op/s | 19149765.378op/s ± 7736.578op/s | 19156461.195op/s | 19166515.235op/s | 19168613.016op/s | 19175126.027op/s | 0.13% | -0.606 | 0.042 | 0.07% | 898.638op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | 6.029µs | 6.040µs ± 0.013µs | 6.036µs ± 0.002µs | 6.039µs | 6.072µs | 6.108µs | 6.111µs | 1.24% | 3.191 | 10.974 | 0.22% | 0.001µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | 163627519.343op/s | 165559247.348op/s ± 365077.935op/s | 165662479.604op/s ± 67797.797op/s | 165723049.364op/s | 165803354.500op/s | 165852427.569op/s | 165872844.241op/s | 0.13% | -3.173 | 10.824 | 0.22% | 25814.908op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | [3.912µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/ | throughput | [255567983.397op/s; 255614536.823op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | [79.499µs; 79.595µs] or [-0.061%; +0.061%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | [12563787.230op/s; 12578999.862op/s] or [-0.061%; +0.061%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | execution_time | [72.990µs; 73.099µs] or [-0.075%; +0.075%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | throughput | [13680429.829op/s; 13700920.059op/s] or [-0.075%; +0.075%] | None | None | None |
| credit_card/is_card_number/37828224631 | execution_time | [3.913µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/37828224631 | throughput | [255536487.570op/s; 255584105.044op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/378282246310005 | execution_time | [69.607µs; 69.700µs] or [-0.067%; +0.067%] | None | None | None |
| credit_card/is_card_number/378282246310005 | throughput | [14347600.557op/s; 14366707.051op/s] or [-0.067%; +0.067%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | execution_time | [52.217µs; 52.227µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | throughput | [19147091.580op/s; 19150684.064op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | execution_time | [6.036µs; 6.038µs] or [-0.019%; +0.019%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | throughput | [165610589.624op/s; 165673342.969op/s] or [-0.019%; +0.019%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | execution_time | [3.912µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | throughput | [255550581.431op/s; 255596209.682op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | [64.471µs; 64.510µs] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | [15501489.372op/s; 15510857.434op/s] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | [58.074µs; 58.119µs] or [-0.038%; +0.038%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | [17206313.539op/s; 17219375.750op/s] or [-0.038%; +0.038%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | [3.912µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | [255573762.230op/s; 255621753.819op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | [54.957µs; 55.037µs] or [-0.073%; +0.073%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | [18170204.238op/s; 18196646.255op/s] or [-0.073%; +0.073%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | [52.221µs; 52.230µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | [19145912.623op/s; 19149435.217op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | [6.038µs; 6.042µs] or [-0.031%; +0.031%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | [165508651.057op/s; 165609843.639op/s] or [-0.031%; +0.031%] | None | None | None |
Group 10
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | 89.508µs | 89.768µs ± 0.172µs | 89.755µs ± 0.054µs | 89.809µs | 89.889µs | 90.182µs | 91.783µs | 2.26% | 8.306 | 93.889 | 0.19% | 0.012µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | [89.745µs; 89.792µs] or [-0.026%; +0.026%] | None | None | None |
Group 11
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 17.776µs | 26.171µs ± 10.254µs | 18.076µs ± 0.217µs | 35.235µs | 44.280µs | 47.884µs | 74.225µs | 310.62% | 0.996 | 0.972 | 39.08% | 0.725µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [24.750µs; 27.593µs] or [-5.430%; +5.430%] | None | None | None |
Group 12
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | 185.168µs | 185.758µs ± 0.308µs | 185.730µs ± 0.221µs | 185.958µs | 186.210µs | 186.309µs | 187.891µs | 1.16% | 1.696 | 9.794 | 0.17% | 0.022µs | 1 | 200 |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | 5322231.159op/s | 5383353.706op/s ± 8909.612op/s | 5384149.294op/s ± 6427.291op/s | 5390203.212op/s | 5394967.456op/s | 5398306.772op/s | 5400509.916op/s | 0.30% | -1.653 | 9.424 | 0.17% | 630.005op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | execution_time | 17.860µs | 17.945µs ± 0.051µs | 17.951µs ± 0.037µs | 17.981µs | 18.018µs | 18.036µs | 18.174µs | 1.24% | 0.352 | 0.795 | 0.29% | 0.004µs | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | throughput | 55022769.717op/s | 55725961.582op/s ± 159202.072op/s | 55707724.509op/s ± 115107.447op/s | 55854974.203op/s | 55969839.502op/s | 55979422.229op/s | 55990040.360op/s | 0.51% | -0.330 | 0.710 | 0.28% | 11257.286op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/good | execution_time | 10.252µs | 10.337µs ± 0.033µs | 10.336µs ± 0.022µs | 10.357µs | 10.390µs | 10.407µs | 10.478µs | 1.37% | 0.337 | 0.826 | 0.32% | 0.002µs | 1 | 200 |
| normalization/normalize_name/normalize_name/good | throughput | 95441303.960op/s | 96743963.181op/s ± 312718.887op/s | 96752204.446op/s ± 205132.704op/s | 96968201.042op/s | 97211989.787op/s | 97468796.742op/s | 97544898.365op/s | 0.82% | -0.311 | 0.769 | 0.32% | 22112.565op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | [185.716µs; 185.801µs] or [-0.023%; +0.023%] | None | None | None |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | [5382118.919op/s; 5384588.492op/s] or [-0.023%; +0.023%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | execution_time | [17.938µs; 17.952µs] or [-0.040%; +0.040%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | throughput | [55703897.706op/s; 55748025.458op/s] or [-0.040%; +0.040%] | None | None | None |
| normalization/normalize_name/normalize_name/good | execution_time | [10.332µs; 10.341µs] or [-0.045%; +0.045%] | None | None | None |
| normalization/normalize_name/normalize_name/good | throughput | [96700623.351op/s; 96787303.011op/s] or [-0.045%; +0.045%] | None | None | None |
Group 13
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | 744.767µs | 745.688µs ± 0.638µs | 745.581µs ± 0.234µs | 745.871µs | 746.582µs | 747.241µs | 751.255µs | 0.76% | 4.235 | 30.884 | 0.09% | 0.045µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | [745.600µs; 745.777µs] or [-0.012%; +0.012%] | None | None | None |
Group 14
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | 4.183ms | 4.187ms ± 0.007ms | 4.186ms ± 0.001ms | 4.187ms | 4.190ms | 4.193ms | 4.279ms | 2.23% | 12.099 | 158.927 | 0.17% | 0.000ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | [4.186ms; 4.188ms] or [-0.023%; +0.023%] | None | None | None |
Group 15
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 1.255µs | 3.214µs ± 1.404µs | 3.003µs ± 0.033µs | 3.038µs | 3.660µs | 13.821µs | 14.829µs | 393.76% | 7.343 | 55.187 | 43.57% | 0.099µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [3.019µs; 3.408µs] or [-6.054%; +6.054%] | None | None | None |
Group 16
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | 13.928ms | 13.992ms ± 0.030ms | 13.988ms ± 0.015ms | 14.004ms | 14.040ms | 14.111ms | 14.138ms | 1.07% | 1.797 | 5.491 | 0.22% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | [13.988ms; 13.996ms] or [-0.030%; +0.030%] | None | None | None |
Group 17
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | 3.123ms | 3.157ms ± 0.021ms | 3.150ms ± 0.009ms | 3.163ms | 3.202ms | 3.217ms | 3.248ms | 3.09% | 1.390 | 1.905 | 0.67% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | [3.154ms; 3.160ms] or [-0.093%; +0.093%] | None | None | None |
Group 18
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | 34.649µs | 35.015µs ± 0.522µs | 34.794µs ± 0.073µs | 34.943µs | 36.021µs | 36.122µs | 38.404µs | 10.38% | 2.418 | 8.168 | 1.49% | 0.037µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | [34.943µs; 35.088µs] or [-0.207%; +0.207%] | None | None | None |
Group 19
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 7b48a91 | 1772814216 | cbeauchesne/final_status |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | 48.845ms | 49.109ms ± 0.947ms | 48.960ms ± 0.044ms | 49.019ms | 49.330ms | 51.347ms | 60.950ms | 24.49% | 10.573 | 123.737 | 1.92% | 0.067ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | [48.977ms; 49.240ms] or [-0.267%; +0.267%] | None | None | None |
Baseline
Omitted due to size.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1681 +/- ##
==========================================
- Coverage 71.17% 71.03% -0.15%
==========================================
Files 427 427
Lines 62810 62962 +152
==========================================
+ Hits 44705 44724 +19
- Misses 18105 18238 +133
🚀 New features to boost your workflow:
|
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
|
How can we test this? |
| report_paths: "target/nextest/ci/junit.xml" | ||
| check_name: "[${{ matrix.platform }}:${{ matrix.rust_version }}] test report" | ||
| include_passed: true | ||
| - name: Add final_status property |
There was a problem hiding this comment.
Instead of adding a new step with a new way of modifying the junit.xml file, would it make more sense to incorporate it into the existing script we have that includes unit tests?
There was a problem hiding this comment.
Definitly. I confess that since I need to do so for 17 repos (and maybe more), I went for a solution that works everywhere and use it regardless of the existing scripts.
But if you'd like to see an implementation in rust, I can give it a try, but I will very probably need your help, since I never write a single line of rust (Claude, be ready 😄 )
Let me know what you prefer !
There was a problem hiding this comment.
If this is a blocker for you, I'm ok merging it as is and we can follow up later to merge it into the existing script. I can probably handle making the change.
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
c37a7dd
into
main
# What does this PR do? Bump to 29.0.0 [feat(profiling)!: add Tracepoint sample type](#1676) [feat(obfuscation/redis): Reach feature parity on redis obfuscation [APMSP-2668]](#1632) [fix(sidecar): Handle backpressure more gracefully](#1682) [feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627]](#1630) [chore: exclude libdatadog from ADMS auto generated PRs for dependency updates](#1688) [chore(ci): run crashtracking ffi example tests in CI](#1687) [fix(crashtracking): use libunwind to unwind frames](#1663) [feat: publish tracer metadata as OTel process ctx](#1658) [ci: run thread count test in own process](#1693) [feat(obfuscation/json): Init json obfuscation [APMSP-2665]](#1635) [chore(ci): add final_status property on junit XML [APMSP-2610]](#1681) [refactor(data-pipeline-ffi): move macro definitions for better reuse](#1699) [fix(obfuscation/memcached): fuzzing fix](#1695) [ci: replace use of cargo cross for centos7 tests](#1675) #1702 (comment) [feat(stats_exporter)!: add process tags to CSS payloads](#1709) # Motivation I mainly am creating this to use the new unwinding from ucontext for crashtracking # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. [APMSP-2668]: https://datadoghq.atlassian.net/browse/APMSP-2668?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-2665]: https://datadoghq.atlassian.net/browse/APMSP-2665?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: gyuheon.oh <gyuheon.oh@datadoghq.com>
What does this PR do?
Add a
<property name="dd_tags[test.final_status]" value="pass" />on every test case reported through junitMotivation
Some repositories intentionally use mechanisms that ignore test failures (there are valid reasons to still execute those tests behind the scenes).
In these cases, the reported test status cannot be relied upon by Test Optimization to monitor repository health or trigger team notifications.
To ensure a consistent and unified approach, we will instead use final_status — the property specifically designed and used by the Test Optimization integration for this purpose.
Additional Notes
I'm using the not-that-ideal XSLT here as this change, because it must be done over 17+ repos, and it's a workaround until this gets integrated into a better place, such as datadog-ci or the backend.
We're trying to work with the team on this, please have a look on this RFC for more context.
How to test the change?
No more
N/Aon@test.final_statuson the last commit of this query